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1 

DIGITAL WATERMARKING 

FIELD OF INVENTION 

The present invention relates to digital watermarking of 
data including image, video and multimedia data. 
Specifically, the invention relates to the insertion and extrac- 
tion of embedded signals for purposes of watermarking, in 
which the insertion and extraction procedures are repeatedly 
applied to subregions of the data. When these sub regions 
correspond to the 8x8 pixel blocks used for MPEG and 
JPEG compression and decompression, the watermarking 
procedure can be tightly coupled with these compression 
algorithms to achieve very significant savings in computa- 
tion. 

BACKGROUND OF THE INVENTION 

The proliferation of digitized media such as image, video 
and multimedia is creating a need for a security system 
which facilitates the identification of the source of the 
material. 

Content providers, i.e. owners of works in digital data 
form, have a need to embed signals into video/image/ 
muhimedia data which can subsequently be detected by 
software and/or hardware devices for purposes of authenti- 
cating copyright ownership, control and management. 

For example, a coded signal might be inserted in data to 
indicate that the data should not be copied. The embedded 
signal should preserve the image fidelity, be robust to 
common signal transformations and resistant to tampering. 
In addition, consideration must be given to the data rate that 
can be provided by the system, though current requirements 
are relatively low — a few bits per frame. 

In VS. patent application Ser. No, 08/534^94, filed Sep. 
28, 1995,. entitled "Secure Spread Spectrum Watermarking 
for Multimedia Data" now abandoned and assigned to the 
same assignee as the present invention, which is incorpo- 
rated herein by reference, there was proposed a spread 
spectrum watermarking method which embedded a water- 
mark signal into perceptually significant regions of an image 
for the purposes of identifying the content owner and/or 
possessor. A strength of this approach is that the watermark 
is very difficult to remove. In fact, this method only allows 
the watermark to be read if the original image or data is 
available for comparison. This is because the original spec- 
trum of the watermark is shaped to that of the image through 
a non-linear multiphcative procedure and this spectral shap- 
ing must be removed prior to detection by matched filtering 
and the watermark is inserted into the N largest spectral 
coefBcients, the ranking of which is not preserved after 
watermarking. Thus, this method does not allow software 
and hardware devices to directly read embedded signals. 

In an article by Cox et al., entitled "Secured Spectrum 
Watermarking for Multimedia" available at http:// 
www, neci.nj.com/tr/index. html (Technical Report No, 
95-10) spread spectrum watermarking is described which 
embeds a pseucb-random noise sequence into the digital 
data for watermarking purposes. 

The above prior art watermark extraction methodology 
requires the original image spectrum be subtracted firam the 
watermark image spectrum. This restricts the use of the 
method when there is no original image or original image 
spectrum available. One application where this presents a 
significant difficulty is for third party device providers 
desiring to read embedded information for operation or 
denying operation of such a device. 
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In U.S, Pat. No. 5,319,735 by R. D. Preuss et al entiUed 
"Embedded Signalling" digital information is encoded to 
produce a sequence of code symbols. The sequence of code 
symbols is embedded in an audio signal by generating a 

5 corresponding sequence of spread spectrum code signals 
representing the sequence of code symbols. The ftcquency 
components of the code signal being essentially confined to 
a preselected signaling band lying within the bandwidth of 
the audio signal and successive segments of the code signal 

10 corresponds to successive code symbols in the sequence. 
The audio signal is continuously frequency analyzed over a 
frequency band encompassing the signalling band and the 
code signal is dynamically ffitered as a function of the 
analysis to provide a modified code signal with frequency 

15 component levels which are, at each time instant, essentially 
a preselected proportion of the levels of the audio signal 
firequency components in corresponding frequency ranges. 
The modified code signal and the audio signal are combined 
to provide a composite audio signal in which the digital 

20 information is embedded. This component audio signal is 
then recorded on a recording medium or is otherwise sub- 
jected to a transmission channel. Two key elements of this 
process are the spectral shaping and spectral equalization 
that occur at the insertion and extraction stages, re^ectively, 

25 thereby allowing the embedded signal to be extracted with- 
out access to the unwatcrmarked original data. 

In U.S. patent application Ser. No. 08/708,331, filed Sep. 
4, 1996,entided "A Spread Spectrum Watermark for Embed- 
ded Signaling'' by Cox; now U.S. Pat. No. 5,848,155 and 

30 incorporated herein by reference, there is described a 
method for extracting a watermark of embedded data from 
watermarked images or video without using an original or 
unwatermarked version of the data. This work can be viewed 
as an extension of the original work of Preuss et al from the 

35 audio domain to images and video. 

This method of watermarking an image or image data for 
embedding signaling requires that the DCT (discrete cosine 
transform) and its inverse of the entire image be computed. 
There are fast algorithms for computing the DCT in N log 

^ N time, where N is the number of pixels in the image. 
However, for N=5 12x512, the computational requirement is 
still high, particularly if the encoding and extracting pro- 
cesses must occTir at video rates, i.e. 30 frames per second. 
This method requires approximately 30 times the computa- 
tion needed for MPEG-Il decompression. 

One possible way to achieve real-time video watermark- 
ing is to only watermark every N^ frame. However, content 
owners wish to protect each and every video frame. 
Moreover, if it is known which frames contain embedded 
signals, it is simple to remove those frames with no notice- 
able degradation in the video signal. 

In U.S. patent application Ser. No. 08/715,953, filed Sep. 
19, 1996,entided "Watermarking of Image Data Using 
MPEG/SPEG Coefficients" by Cox, and incorporated herein 
by reference, there is described an alternative method, which 
is to insert the watermark into nxn blocks of the image 
(subimages) where n«N. Then the computation cost is 

60 ^rt \ogn = N logn. 

For N=»512x512=2^® and n=8x8=2^, the asymptotic sav- 
ing is only a factor of 3. However, empirically the cost of 
65 computing the DCT over the entire image may be signifi- 
cantly higher when cache, loop unfolding and other effi- 
ciency issues are considered. Thus, the practical difference 
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may approach a 30 fold savings. More importantly, if the FIG. 6 is a graphic representation of rotation of PN 

block size is diosen to be 8x8, i.e. the same size as that used sequences; 

for MPEG image compression, then it is possible to tightly FIG. 7 is a graphical representation of an 8x8 block 

couple the watermark insertion and extraction procedures to shown the spatial relation of averaged terms; 

those of the MPEG compression and decompression algo- S FIG. 8 is a schematic block diagram of a method for 

rithms. Considerable computational saving can then be inserting watermarks in accordance with the present inveo- 

achieved since the most expenses computations relate to the tion; and 

calculation of the DCT and its inverse and these steps are FIG. 9 is a schematic block diagram of a method for 

already computed as part of the compression and decom- extracting watermarks in accordance with the present inven- 

pression algorithm. The incremental cost of watermarking is lO tion. 

then very small, typically less than 5% of the computational DETAILED DESCRIPTION 

requirements associated with MPEG. . . ^ , ^.^r. ^ . , ^ ■ 

„ . ^ , Referrmg now to the figures, and FIGS. 1 through 4 m 

TTic present mvenUon unproves the rebabikty of the jhere is shown schematic block diagrams of a 

invention described in the 08/715.953 apphcauon, now ^ ^^^^^ j^^j^ watermarks in 

pendmg by storing watermark information into submiages, is j j^j^^,^ j 

and extracting watermark information firom subimages, in a i r » ■ j • r j . 

manner different from that described earUer. . ^ following desaiption. reference may be made o 

miage data or miages. While the mvention has appucability 

SUMMARY OF THE INVENTION ^ image data and images, it will be understood that the 

2Q teachings herein and the invention itself are equally appli- 

The present invention improves the rehability of the prior cable to video, image and multimedia data and the term 

systems by systematically varying the order in which water- "image" and "image data" will be understood to include 

mark signal components are inserted into each subimage, by these terms vi^ere applicable. As used herein, "watermark" 

inserting only part of the watermark signal into each will be understood to include embedded data, symbols, 

subimage, and, during watermark detection, by combining 25 images, instructions or any other identifying information, 

the watermark signals found in groups of subimages to In the following description, reference is made to proce- 

recoostruct the original watermark signal before testing for dures described in U.S. patent application Ser. No. 08/534, 

correlation with any predefined watermarks. 894 for inserting and extracting or detecting a watermark in 

For detection, a reverse transformation is applied to each images as INSERT-ORIGINAL and EXTRACT- 

subimage to reconstruct the watermark information that was 30 ORIGINAL, respectively. Reference is made to procedures 

stored in that subimage. The resulting signals are then described in U.S. patent application Ser. No. 08/708331 

averaged together to reconstruct the whole watermark, and filed Sep. 4, 19964iow U.S. Pat. No. 5,848,155 for inserting 

to reduce noise. Finally, this reconstructed watermark is and extracting or detecting watermarks in images as 

compared against a predefined set of watermark signals to INSERT- WHOLE and EXTRACT- WHOLE, respectively, 

determine which one was inserted into the image. 35 And reference is made to procedures described in U.S. 

Aprincipal object of the present invention is therefore, the P*^^°^ application Ser. No. 08/715,953 for inserting and 

provision of inserting a subset of a watermark into a subset extracting or detecting watermarks in images as INSERT- 

of subregions of data to be watermarked. MPEG-A and EXTRACT-MPEG-A, respectively. 

Another object of the invention is the provision of a ,J}^J^ '^^^ ^ schematic block diagram of INSERT- 

digital watermarking system in which a watermark is WHOLE procedure for msertmg watermarks mto miages 

extracted by averaging the watermarked signal from subre- The watermark signal, m the form of a fimte sequence of 

gions of watermarked data, and then correlating the resulting ^>^^^ ^ alphabet is provided as an mput to 

signal to determine the watermark. ^"""^ correction encoder 10 which transforms this 

^^^^1^ ^- . ^ ^ ■ ^ sequence mto another sequence that contains redundant 

A fiirther object of the mvenUon is the provision of a information. The output of encoder 10 is provided to a 

digital watennarkmg system m which the watermark is ^tiich maps each symbol of the encoded 

composed of two porUons, a venfication portion and a ^.t^miark into a pre-specified pseudo-random noise (PN) 

synchromzation poruon, m order to miprove watermark ^^^^^^ PN-mapper 11 is provided to a 

extraction reliabihty ... spectral transformer 12, which converts the pseudo-random 

Further and still other objects of the invention will noise sequence into the frequency domain. The conversion 

become more clearly apparent when the following descrip- preferably is by discrete cosine transform (DCI), however, 

tion is read in conjunction with the accompanying drawing. fast fourier transform, wavelet type decomposition and the 

like may also be used for frequency conversion. 

BRIEF DESCRIPTION OF THE DRAWINGS Concurrently, the data to be waterrnarkcd is provided to 

FIG. 1 is a schematic block diagram of watermark inser- 55 another spectral transformer 13. The outputs of the two 

tion procedure- spectral transformers 12 and 13 are then provided as inputs 

FIG. 2 is a schematic block diagram of a watermark " '^'^^^t ""^''f* fT'" 

. . _j .L * u- r»u ties of the pseudo-random noise codes from spectral trans- 

insertion procedure m accordance with the teachmgs of the ^ ; , . . . jj j . • 

resent invention- former 12 to mask the watermark when added to the image 

preseo mven on, ^^^^ spectrally transformed data to be watermarked, 

HG, 3 IS a schematic block diagram of a watermark spectral transformer O, is also provided as an input to 

extraction procedure; a delay 15. The output of the spectral shaper 14 is then added 

FIG. 4 is a schematic block diagram of a watermark to the output of delay 15 at a summer 16. The summer output 

extraction procedure in accordance with the teachings of the is subject to an inverse transform 17. The result of the 

present invention; 65 inverse transform is watermarked data. 

FIG. 5 is a graphic representation of a zigzag pattern INSERT-MPEG-A differs from INSERT- WHOLE by scg- 

usefiil for vectorizing subimages; menting the data to be watermarked into multiple blocks. 
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such as 8x8 pixel subimages or subregions. Eadi block of into blocks by data segmenter 40, which corresponds to the 

data thcQ has the watermark inserted according to the above data segmenter 24 used during the insertion procedure in 

described^ method. That is, for each 8x8 subimage or FIG. 2, Each of the data blocks is provided to a respective 

subregion, a pseudo-random number (PN) sequence is spectrum normalizer 41a, 41b, etc. to produce a signal 

inserted into the OCT coefiScients after suitable spectral 5 rescmbhng the subwatcrmark that was inserted into the 

shaping. The procedure is repeated for all such subimages or respective data block. These inserted subwatermark signals 

subregions. The size of the subimage or subregion is pref- ^rc then used as inputs into a watermark combiner 42. In the 

erably 8x8, but it can be of other sizes, such as 2x2, 3x3, 4x4 ^^^^^^^ 43, parts of the watermark that appear redundanUy 

or 10x10. several subwatermarks are averaged together to reduce 

FIG. 2 shows a schematic block diagram of a watermark ^^q^^ ^^^^^ watermark combiner 42 is provided 

insertion procedure in accordance with teachings of the ^ ^jput to a symbol separator 43 which divides the 

present invention. The watermark signal is processed into a watermark into parts, each of which corresponds to one 

noise spectrum signal by the error correction encoder 20, the symbol from the encoded watermark signal (the output of 

PN mapper 21, and the spectral transformer 22, in the same ^^ror correction encoder 20 in FIG. 2). 

"^T^JM^f^""^^^^^ ^y^^^^ ^P^^^^ 43 are provided as inputs 

unlike INSERT.WHOLE or INSERT-MPEG-A, the water- ^ ^ ^^^^^ watermark identifiers 44a, 446 etc. each of 

mark IS then used as an input to a watermark segmenter 23 which includes of a bank of correlators and a decision 

which systemaucally separates the watermark mto several ^ ^ 3 ^ watermark 

subwatermarks. Any porUon of Uie onginal watermark identifiers are symbols from the alphabet used in the original 

might appear reaundantly la several of the resulbng subwa- ,„ j j . i - i -n. -j .-c j u i 

&" rr . , J ■ .2" encoded watermark signal. The identified symbols are reas- 

termarks. Concurrently, the data to be watermarked IS used uij-. i. jj. lu.i. ui 

.... ■" . ... • .1. J • sembled mto a complete encoded watermark by the symbol 

as an mput to data segmenter 24, which segments the data , . 11 .u i.- j j . 1 • 

.,, L ^nt- • combiner 45. Fmally, the resulting encoded watermark is 

into blocks or subregions, such as 8x8 subimages, asm . jju.u .j^ 

.^T.^r^ni^ .^l ^-l l . l dccoded by thc crTor conector 46. 

INSERT-MPEG-A. Each of the subwatermarks output by ^ . . 

the watermark segmenter 23 is then inserted into a data „ , The insertion and extraction procedhires wiU now be 

block by one of the watermark iiKerteis 25a, 256. etc. The '^f I" INSERT-ORIGINAL and 

procedure used by the watermark inserters 25a. 2Sb, etc.. is EXTRACT-ORIGINAL the object is to embed a single PN 

the same procedure described connection with watermark ^""^l random number) sequence mto an unage when the 

inserter 18 in FIG. L That is, each subwatermark is added .^"8= is available at the time of extraction. The 

into a spectrally transformed data block after spectral ,„ information assocmted with ttie PN sequence is assumed to 

shaping, and the resulting data is then transformed back into ^ f daubase together with the ongmal image and 

the spatial domain. FinaUy, the watermarked data blocks °[ ^ embedded watermark. The loca- 

from the watermark inserters 2Sfl,25*, etc., are assembled by watermarked components ha^ to be recorded 

data combiner 26 to produce watermarked data. ^^"^ the miplementatiOD approximated the N perceptu- 

™^ i_ ally most significant regions of the watermark by the N 

FIG. 3 shows a schematic block diagram of the 35 , , «- • * xi .u- 1 • * • • * 

CYTO Ar-r worn c „«f^ largest cocfiBcicnts. However, this ranking was not invanant 

EXJRACl-WHOLb procedure. The watermarked image, ^ ~i ^ , . ti. kt 1 * re • * 

1.* J- J * • c . J . .1 to the watermarking process. The N largest coeflBcients may 

video or mulumedia data is first used as mput mto a spectral . j-a- . • ^- * 1 *i_ i_ r • 

. J • 1 _f J * 1 be different after msertmg the watermark than before insert- 

normalizer 30 to undo any previously performed spectral • * , 

shaping. If the data contains a watermark, then the output of ^ 

the spectral normaHzer 30 will resemble the spectral trans- 40 , ^'''''^ problem, the present invention 

formation of tiie PN coding of that watermark (the signal P^^^^s a watermark m predetermmed locations of the 

that was input to the spectral shaper 14 in FIG. 1). The spectrum, typicaUy the first N coefficients. However, any 

output of the spectral normalizer 30 is then used as an input predetermmed locations could be used, though such loca- 

to several correlators 31a, 31b, etc., which test the water- ^""^^ should belong to the perceptually significant regions of 

mark with the PN codes used to represent the various 45 ^l^e sP^^rum if the waterai ark is to survive common signals 

symbols that the encoded watermark might contain (i.e. each transformauons such as compression, scaling, etc. 

correlator tests for one PN code that is used to encode a More generally, the information to be embedded is a 

symbol by the PN mapper 11 of FIG, 1). The outputs of the sequence of m symbols drawn from an alphabet A (e.g. the 

correlators 31a, 31b, etc., are used as inputs to a decision binary digits or the ASCII symbols). This data is then 

circuit 32, which determines the most likely sequence of 50 supplemented with additional symbols for error detection 

symbols. Finally, this sequence is corrected by an error correction. Each symbol is then spread spectrum 

corrector 33, which performs the inverse coding tiiat was modulated, a process that maps each symbol into a unique 

performed by tiie error correction encoder 10 in FIG. 1. The PN sequence known as a chip. The number of bits per chip 

result is Uie extracted watermark. ^ preset - the longer the chip length, the higher the detected 

In EXTRACT-MPEG-A, the data from which a water- 55 signal-to-noise ratio will be, but this is at the expense of 

mark is to be extracted is first segmented into several blocks, signahng bandwidth. 

such as 8x8 subimages, exacUy as in INSERT-MPEG-A. The power spectrum of the PN sequence is white, i.e. flat. 

The signal from each subimage is then normalized and used and is therefore shaped to match that of the "noise", i.e. the 

as input into a bank of correlators similar to the correlators image/video/audio/or multimedia data into which the water- 

31fl, 31b, etc. in FIG. 3. The output from the correlators is eo ^^^^ ^ }^ ^ embedded. It is tiiis spectral shaping that must 

then averaged with the outputs of corresponding correlators modified from the prior methods so that the extraction 

from other subimages, and the resulting average correlations process no longer requires the original image. To do this, 

are used as inputs into the decision circuit 32 for subsequent each coeflSdent of tiie watermarked spectrum is scaled by 

processing as described above. local average of the power in the image spectral coef- 

FIG- 4 shows a schematic block diagram of a watermark 65 ^^^^^ ^^^^"^ coefficient itself, i.e. 
extraction procedure in accordance with the teachings of the 

present invention. The watermarked data is first segmented //"/r+ctavgd/^Otv;. (1) 
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The averaging Is the averaging of the absolute coefficient found to improve the detection response and significandy 

values and not the coefficient values themselves. This is reduced the computation requirement associated with each 

effectively estimating the average power present at each blodc. 

frequency. Other averaging procedures are possible, for In practicing the present invention preferably there is a 

example, averaging over several frames or average of local 5 unique PN sequence for each symbol in the alphabet. The 

neighborhoods of 8x8 blocks. method is relatively robust to cKpping since the detector 

This average may be obtained in several ways. It may be output reduces linearly with the quantity of 8x8 subimage 

a local average over a two dimensional region. Alternatively, blodcs in the image. For DVD (digital video disk) embedded 

the two dimensional spectrum may be sampled to form a one signaling for APS (analog protection system) and CGMS 

dimensional vector and a one dimensional local average may 10 (copy generation management system), there woiild be a 

be performed. One dimensional vectorization of the two total of 8 or 16 PN sequences. 

dimensional 8x8 DCT coefficients is ahready performed as The number of 8x8 blocks in a 512x512 image is 4096, 

part of MPEG II. The average may be a simple box or suggesting that significantly more than one of 16 symbols 

weighted average over the neighborhood. can be embedded in an image or video frame. Assume, for 

For video data, temporal averaging of the spectral coef- 15 example, that it is desired to embed 1 out of 128 symbols in 

ficients over several frames can also be applied. However, an image. It is necessary to perform 128 parallel oorrela- 

since several frames are needed for averaging at the spectral tions. This is computationally tractable but hardware imple- 

normalization stage of the extractor, the protection of indi- mentations of each correlation become more complex. An 

vidual video frames taken in isolation may not be possible. alternative method is to only use two binary symbols. It may 

For this reason, the present invention treats video as a very 20 be preferable to associate more than one PN sequence with 

large collection of still images. In this way, even individual each of the two binary symbols or bits in order to increase 

video frames are copy protected. the difficulty of intentionally removing the watermark. In 

In order to extract the watermark, it is necessary to this case, there are only two correlators and a binary string 

perform the spectral normalization, in which the previously may be embedded into the image. The raw bit error rate wlQ 

performed spectral shaping procedure is inverted. In the 25 be very high due by the low detector output However, this 

present invention, the original unwatermarked signal is not can be reduced to acceptable levels by using error correcting 

available. Thus, the average power of the frequency codes, such as Reed-Solomon (RS). RS codes are robust to 

coefficients, avg(|f,^), is approximated by the average of the burst error which may occur because of clipping of the 

watermarked signal, i.e. avg(|f/|) image. Other error correcting codes may also be used. 

30 When using this method, it is necessary for the receiver to 

avg(l/;j>-^,(j/;'D (2) know the start location of the encoded block. The start 

_ . . . , . /ir r ( location may not be obvious, particularly when the image 

-nus IS approximately true smce aavg(lf.^)W, where J^^^^ ^^^ ^^ Hov^^^^cl convention syt 

W, IS the watermark component, and ais a constant typically ^tironizing methods can be used; such as preceding each 

m the range between 0.1 and 0.01. ^ ^^^^ ^ t^^l symbols. 

The normalization stage then divides each coefficient (f- ) t,. . i t-ooLii-T*^ 

. . , . , t . t /If-, IX - V To insert a watermark, each 8x8 block is treated as an 

m the received signal by the local average avg ( r-' ) m the • j- j i u- i • t-l r\r^ c*l l- 

•hhhfvi & & yi i\/ mdividual subimage or subregion. The DCT ot the subimage 

^^^Th ^ ' is then computed and the two dimensional DCT is vectorized 

^ ^' in the zigzag pattern shown in FIG. 5, although other 

, ^ pattems are also possible. These two stages constitute most 

/; _ fi^-aavgum (3) calculations but are part of the MPEG encoding 



avgUfi I) avgi\fi\) process. Next, a PN noise sequence {w^. . . w„} is inserted 

fi ^fjyj. into the DCT coefficients using Equation 1 as before. The 

av^l/'D ' length of the PN sequence cannot exceed 64 (in an 8x8 

45 block) and is typically much shorter, in the range of 11 to 25 



The first term, on the right hand side (RHS) ot Equation " ""^L a single code is to be inscrt«l into the iinage, 

/OX ^ or- \ / 1 ^^jjj jjjg s^m^ PN sequence is inserted mto each of the 

^ 720 x 480/64 -5400 blocks. However, a variation may be 

performed at this point in the procedure. Within each row of 

ayg!\f'\y blocks, the PN sequence is cychcally rotated by one fre- 

' quency coefficient prior to insertion in the subsequent block. 

Similarly, the PN sequence is cychcally rotated by one 

is considered a noise term. This term was not present in the frequency coefficient at the start of each new row. FIG. 6 

system described in U.S. patent application Ser. No. 08/534, illustrates an order of rotations. 

894, because access to the unwatermarked coefficients 55 The purpose of these rotations or shifts is to improve the 

allowed this term to be removed. The second term aW^ is the response of the watermark extraction stage. Earlier cxperi- 

original watermark signal which can now be detected using ments revealed that certain DCT coefficients were more 

conventional correlation. difficult to estimate than others. The location at these coef- 

If the watermark is extracted from any single 8x8 block, ficients varied from image to image. However, within an 

the detector reliability is very low. If, however, the water- 60 image, the coefficient could be consistently poor, 

marks extracted from each 8x8 block are first added together Consequently, without shifting, one or more of the estimated 

and the averaged watermark is then applied to the correlator, watermark coefficients could be significantly degraded rela- 

then a very strong and imambiguous response is obtained. tive to the other watermark coefficients, thereby reducing the 

This differs from the method described in U.S. patent detector performance. Conversely, shifting significantly 

application Ser. No. 08/715,953 in which correlation 65 reduces the effect a poor DCT coefficients has on a single 

occurred within each block and the output from each corr- watermark coefficient and the detector performance is mark- 

e la tor was averaged together. The present invention was edly improved. Note that any cyclic pattern can be used. 
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Further modifications are useful once rotation of the 
watermark has been introduced. First, the length of the 
watermark may now be significantly greater than 64 .Then, 
for each block only a small subset of the watermark (say 
five) cocfGcients is inserted into the first five DCT cocfiB- 5 
cicnts (excluding the d.c. term). Because of the rotation, a 
different subset of the watermark is inserted into neighbor- 
ing 8x8 blocks. Finally, having completed the watermark 
insertion, the MPEG encoder is able to proceed with the 
subsequent stages of compression. lo 

Note that the watermark may also be inserted after the 
MPEG quantization stage to reduce distortion of the water- 
mark. MPEG-2 performs a convenient one dimension vec- 
torization called "zigzagging", which allows a simple 3x1 
box average to be performed on the coefficients (exchiding 15 
the d.c. term). 

In practice, performance was improved if the averaging is 
performed using the 2 four-connected coefficients closest to 
the d.c. term, as illustrated in FIG. 7, i.e. the two coefiBcients 
above and to the left. 20 

Watermark detection begins by first extracting the PN 
noise sequence from each 8x8 block using Equation l.For 
each block, the PN sequence is then cyclically shifted in the 
opposite direction by one frequency coefficient, and the 
average over all the blocks is then computed. In practice, this 25 
process can be computed incrementally and does not require 
temporary storage of all the extracted watermarks. A 
weighted averaging can also be applied, where the weights 
are determined based on their susceptibility to common 
signal transformations such as low pass filtering. Finally, the 
average watermark is compared with the original PN 
sequence via correlation. The reason for shifting the water- 
mark in the column direction may now be apparent. If the 
image is clipped on an arbitrary block boundary, then the 
computed average watermark will simply be rotated by an 
amount that is a function of the relative location of the 
clipped portion of the image. Correlation can then be per- 
formed on all permutations (typically 11 to 25) of the 
watermark. The output from the correlator with the maxi- 
mum value is then used for decision purposes. The extrac- 
tion stage is depicted in FIG. 4. Taking the maximum 
correlator output over all rotations of the watermark can 
cause the decision circuitry to be noisy. To improve this, the 
watermark is broken into two pieces; a synchronization 
portion is of length K and a verification portion is N-K. 45 
Then, when the watermark is extracted as before, correlation 
is first performed only on all rotations of the synchronization 
portion of this watermark. The maximum correlation output 
is noted, then the verification portion of the watermark is 
rotated by the corresponding amount and a second correla- 50 
tion is performed on the verification portions of the water- 
marks. This process significantly improves the overall reli- 
ability of the system. In the course of experimentation, it was 
noticed that some watermarks performed better than others 
on the same imagery. This was caxised by variation in the 55 
correlation statistics between the synchronization and veri- 
fication portions of the watermark. Ideally, the two portions 
should have very low correlations. However, in several cases 
where watermarks performed poorly, it was traced to unex- 
pected correlations between the two portions. 60 

The present invention provides a modification to digital 
watermarking methods in which the original data is required 
for watermark extraction thereby enabling watermarking 
extraction in the absence of an unwatermarked or original 
data. The present invention preferably uses MPEG/JPEG 65 
coefficients. An image is divided into typically 8x8 block 
subimages or subregions and each subimage is processed 
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and the results are combined to derive the extracted water- 
mark. The result is extraction of the watermaric with very 
high confidence. 

While the above invention describes improvements to the 
prior-art INSERT-WHOLE, INSERT-MPEG-A, 
EXTRACT-WHOLE, and EXTRACT-MPEG-A algorithms, 
it should be apparent to anyone skilled in the art that the 
same improvements may be applied to any algorithm for 
inserting and extracting watermarks in image data. This 
more general view of the present invention is shown in 
FIGS. 8 and 9. 

FIG. 8 shows a schematic block diagram of the general 
method for inserting watermarks. This general method 
makes use of a non-block-based watermark insertion 
algorithm, whidi shall be referred to hereafter as the "base 
insertion algorithm". The watermark encoder 80 converts 
the watermark into a form appropriate for the base insertion 
algorithm. If the base insertion algorithm is that shown in 
FIG. 1, for example, then the watermark encoder 80 corre- 
sponds to the watermark encoder 19, which comprises the 
error correction encoder 10, the PN mapper 11, and the 
spectral transformer 12. However, if a different base inser- 
tion algorithm is to be used, then the watermark encoder 80 
may perform a different transformation of the watermark. 
The encoded watermark signal from watermark encoder 80 
is provided as an input to watermark segmenter 81, which 
divides the watermark into a set of subwatermarks. Any 
portion of the original watermark might appear redundanUy 
in several of the resulting subwatermarks. The data to be 
watermarked is provided as an input to data segmenter 82, 
which divides the data into subregions. Each subwatermark 
is inserted into a respective data subregion by a watermark 
inserter 83ii, 836, etc. The watermark inserters implement 
the base insertion algorithm, so, if the base insertion algo- 
rithm is that shown in FIG. 1, then each watermark inserter 
S3a, 836, etc., corresponds to the watermark inserter 18, 
which comprises a spectral transformer 13, a spectral shapcr 
14, a delay 15, a summer 16, and an inverse transform 17. 
However, if a different base insertion algorithm is to be used, 
then the watermark inserters S3a, 83b, etc., may employ a 
different method of inserting subwatermarks into the subre- 
gions of the data to be watermarked. The outputs from the 
watermark inserters are assembled in data combiner 84 to 
provide watermarked data. 

FIG. 9 shows a schematic block diagram of the corre- 
sponding general extraction algorithm. The algorithm makes 
use of a "base extraction" algorithm that corresponds to the 
base insertion algorithm used in inserting the watermark into 
the data to be watermarked (FIG. 8). The algorithm in FIG. 
9 is substantially the same as the algorithm shown in FIG. 
4, except that, in the general case, the spectrum normalizers 
41fl^ etc. are replaced by watermark extractors 91a, etc., 
which implement the base extraction algorithm. That is, if 
the base insertion algorithm used was the algorithm shown 
in FIG. 1, then the watermark extractors 9ia, etc., in FIG. 9 
will be the spectrum normalizers 41a, etc. in FIG. 4. 

While there has been described and illustrated a system 
for inserting a watermark into and extracting a watermark 
from watermarked data without using an unwatermarked 
version of the data, it will be apparent to those skilled in the 
art that variations and modifications are possible without 
deviating from the broad principles and teachings of the 
present invention which shall be limited solely by the scope 
of the claims appended hereto. 

What is claimed is: 

1. A method for inserting a watermark signal into data to 
be watermarked comprising the steps of: 
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dividing data to be watermarked into a plurality of sub- 
regions; 

computing frequency coefficients of the data to be water- 
marked in each subregion; 

spread spectrum modulating a watermark signal to be 
inserted by mapping the watermark signal into a PN 
(pseudo- random noise) sequence; 

spectral shaping the PN sequence as a function of the 
average power in each frequency coefficient of the data; 
and 

inserting each spectral shaped PN sequence into prede- 
termined coefficients in the data in each subregion. 

2. A method for inserting a watermark signal into data to 
be watermarked as set forth in claim 1, where said inserting 
is performed after the data undergoes MPEG quantization 
processing. 

3. A method for inserting a watermark signal into data to 
be watermarked as set forth in claim 1, where said frequency 
coefficients are DCT (discrete cosine transform) coefficients. 

4. A method for inserting a watermark signal into data to 
be watermarked as set forth in claim 3, where each subre- 
gion is a 8x8 blodc of pixels. 

5. A method for inserting a watermark signal into data to 
be watermarked as set forth in claim 4, where said inserting 
is performed after the data undergoes MPEG quantization 
processing, 

6. A method for inserting a watermark signal into data to 
be watermarked as set forth in claim 1, where each subre- 
gion is a 8x8 blodc of pixels. 

I. A method for inserting a watermark signal into data to 
be watermarked as set forth in claim 6, where said inserting 
is performed after the data undergoes MPEG quantization 
processing. 

8. A method for inserting a watermark signal into data to 
be watermarked as set forth in claim 6, where the frequency 
coefficients of the watermark signal are rotated prior to 
inserting of each spectral shaped PN sequence into the 
subregion. 

9. A method for inserting a watermark signal into data to 
be watermarked as set forth in claim 8, where said inserting 
is performed after the data undergoes MPEG quantization 
processing. 

10. A method for inserting a watermark signal into data to 
be watermarked as set forth in claim 8, where only a subset 
of the watermark signal frequency coefficients is inserted 
into any one subregion. 

II. A method for inserting a watermark signal into data to 45 
be watermarked as set forth in claim 10, where the water- 
mark signal comprises a synchronization portion and a 
verification portion. 

12. A method for inserting a watermark signal into data to 
be watermarked as set forth in claim 11, where said inserting 
is performed after the data undergoes MPEG quantization 
processing. 

13. A method for inserting a watermark signal into data to 
be watermarked as set forth in claim 11, where the synchro- 
nization portion and the verification portion have very little 
correlation between each other. 

14. A method for inserting a watermark sigaal into data to 
be watermarked as set forth in claim 1, where the spectral 
shaping as a function of the average power is typically 3x1 
window of the coefficient obtained from the one- 
dimensional vectorization by zigzagging of two-dimension 
frequency coefficients. 

15. A method for inserting a watermark sigaal into data to 
be watermarked as set forth in claim 1, where the spectral 
shaping is a function of the average power based on the two 
four-connected frequency coefficients closest to the DC 
term. 
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16. A method of exuracting a watermark from water- 
marked data comprising the steps of: 

receiving subnegioas of watermarked data; 

spectrum normahzing the watermarked data as a function 
of the average power in each frequency coefficient of 
the watermarked data in each subregion to generate 
respective normalized signals; 

combining the respective normalized signals frt)m each 
subregion to generate a single watermark; 

correlating the single watermark with predetermined PN 
(pseudo- random noise) sequences corresponding to 
predetermined symbols to provide correlated signals 
for each predetermined PN sequence in each subregion; 

deciding which correlated signal is most likely a current 
symbol; and 

extracting a sequence of most likely current symbols 
corresponding to the watermark. 

17. A method of extracting a watermark from water- 
marked data as set forth in claim 16, where the subregions 
are 8x8 blocks used for MPEG encoding and decoding. 

18. A method of extracting a watermark from water- 
marked data as set forth in claim 17, where said combining 
the normalized signals frx)m each subregion to generate a 
single watermark, including removing the relative rotation 
of the watermark between blocks. 

19. A method of extracting a watermark from water- 
marked data as set forth in claim 18, further comprising 
subsequently reconstructing the watermark from partial 
watermarks inserted into each block. 

20. A method of extracting a watermark from water- 
marked data as set forth in claim 19, further comprising 
weighting the watermark coefficients based on their location 
within the frequency spectrum, where the weighting is a 
function of the susceptibility of each frequency coefficient to 
common signal transformations. 

21. A method of extracting a watermark from water- 
marked data as set forth in claim 16, further comprising 
correlating with all rotational shifts of the extracted water- 
mark and selecting the maximum value. 

22. A method of extracting a watermark from water- 
marked data as set forth in claim 16, further comprising 
correlating with all rotational shifts of a synchronization 
portion of a watermark to determine a maximum value and 
subsequendy rotating a verification portion of the watermark 
by the same amount as the synchronization portion is rotated 
to obtain the maximum value prior to correlating between 
the verification portion and predetermined PN sequences. 

23. A method of extracting a watermark from water- 
marked data comprising the steps of: 

receiving subregions of watermarked data; 

spectrum nomializing the watermarked data as a function 
of the average power in each frequency coefficient of 
the watermarked data in each subregion to generate 
respective normalized signals; 

correlating the respective normalized signals with prede- 
termined PN sequences corresponding to predeter- 
mined symbols to provide correlated signals for each 
predetermined PN sequence in each subregion; 

deciding which correlated signal is most likely a current 
symbol in each subregion for providing an extracted 
symbol stream; 

error correcting the extracted symbol stream; and 

extracting a sequence of most likely current symbols 
corresponding to the watermark. 

24. A method of extracting a watermark from water- 
marked data as set forth in claim 23, where said error 
correction is Reed Solomon error correction. 
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25. A method for inserting a watermark signal into data to 
be watermarked comprising the steps of: 

dividing data to be watermarked into a plurality of sub- 
regions; 

dividing a watermark signal into a plurality of subwater- 
marks where portions of the watermark are contained in 
more than one subwatermark; and 

inserting said plurahty of subwatermarks into said plu- 
rality of subregions. 

26. A method for inserting a watermark signal into data to 
be watermarked as set forth in claim 25, where each sub- 
watermark is inserted into a respective subregion, so that 
each subregion contains at least one subwatermark. 

27. A method for extracting a watermark signal from 
watermarked data comprising the steps of: 

receiving a plurality of subregions of watermark data; 
extracting a subwatermark firom each subregion of said 
plurality of subregions; and 
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combining and averaging the subwatermarks extracted 
from all the subregions to obtain a signal commensu- 
rate with the watermark signal. 
^ 28. A method for extracting a watermark signal from 
watermarked data as set forth in claim 27, further compris- 
ing the steps of: 

dividing the signal commensurate with the watermark 
signal into a plurality of symbol signals; 

10 

correlating each symbol signal with a set of predefined 
signals; 

determining which predefined signal best corresponds to 
each symbol signal; and 

combining the best corresponding predetermined signals 
to generate the watermark signal. 
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